36 research outputs found
Zero Shot Learning for Code Education: Rubric Sampling with Deep Learning Inference
In modern computer science education, massive open online courses (MOOCs) log
thousands of hours of data about how students solve coding challenges. Being so
rich in data, these platforms have garnered the interest of the machine
learning community, with many new algorithms attempting to autonomously provide
feedback to help future students learn. But what about those first hundred
thousand students? In most educational contexts (i.e. classrooms), assignments
do not have enough historical data for supervised learning. In this paper, we
introduce a human-in-the-loop "rubric sampling" approach to tackle the "zero
shot" feedback challenge. We are able to provide autonomous feedback for the
first students working on an introductory programming assignment with accuracy
that substantially outperforms data-hungry algorithms and approaches human
level fidelity. Rubric sampling requires minimal teacher effort, can associate
feedback with specific parts of a student's solution and can articulate a
student's misconceptions in the language of the instructor. Deep learning
inference enables rubric sampling to further improve as more assignment
specific student data is acquired. We demonstrate our results on a novel
dataset from Code.org, the world's largest programming education platform.Comment: To appear at AAAI 2019; 9 page
Variational Temporal IRT: Fast, Accurate, and Explainable Inference of Dynamic Learner Proficiency
Dynamic Item Response Models extend the standard Item Response Theory (IRT)
to capture temporal dynamics in learner ability. While these models have the
potential to allow instructional systems to actively monitor the evolution of
learner proficiency in real time, existing dynamic item response models rely on
expensive inference algorithms that scale poorly to massive datasets. In this
work, we propose Variational Temporal IRT (VTIRT) for fast and accurate
inference of dynamic learner proficiency. VTIRT offers orders of magnitude
speedup in inference runtime while still providing accurate inference.
Moreover, the proposed algorithm is intrinsically interpretable by virtue of
its modular design. When applied to 9 real student datasets, VTIRT consistently
yields improvements in predicting future learner performance over other learner
proficiency models.Comment: 9 pages, 16th International Conference on Educational Data Mining
(EDM'23
Bayesian Decision Trees via Tractable Priors and Probabilistic Context-Free Grammars
Decision Trees are some of the most popular machine learning models today due
to their out-of-the-box performance and interpretability. Often, Decision Trees
models are constructed greedily in a top-down fashion via heuristic search
criteria, such as Gini impurity or entropy. However, trees constructed in this
manner are sensitive to minor fluctuations in training data and are prone to
overfitting. In contrast, Bayesian approaches to tree construction formulate
the selection process as a posterior inference problem; such approaches are
more stable and provide greater theoretical guarantees. However, generating
Bayesian Decision Trees usually requires sampling from complex, multimodal
posterior distributions. Current Markov Chain Monte Carlo-based approaches for
sampling Bayesian Decision Trees are prone to mode collapse and long mixing
times, which makes them impractical. In this paper, we propose a new criterion
for training Bayesian Decision Trees. Our criterion gives rise to BCART-PCFG,
which can efficiently sample decision trees from a posterior distribution
across trees given the data and find the maximum a posteriori (MAP) tree.
Learning the posterior and training the sampler can be done in time that is
polynomial in the dataset size. Once the posterior has been learned, trees can
be sampled efficiently (linearly in the number of nodes). At the core of our
method is a reduction of sampling the posterior to sampling a derivation from a
probabilistic context-free grammar. We find that trees sampled via BCART-PCFG
perform comparable to or better than greedily-constructed Decision Trees in
classification accuracy on several datasets. Additionally, the trees sampled
via BCART-PCFG are significantly smaller -- sometimes by as much as 20x.Comment: 10 pages, 1 figur